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Report  Title 

SYSTEM  RELIABILITY-BASED  DESIGN  OPTIMIZATION  UNDER  INPUT  AND  MODEL  UNCERTAINTIES 

ABSTRACT 

For  reliability -based  design  optimization  (RBDO),  sensitivity  analysis  capability  is  a  major  bottleneck  for  broader  use  of  RBDO  methods  in 
multidisciplinary  M&S  applications  of  Army  complex  physical  systems.  To  overcome  this  bottleneck,  a  sequential  sampling-based  dynamic 
Kriging  (DKG)  method  is  developed.  The  DKG  method  has  been  integrated  with  the  lowa-RBDO  software  system  that  is  developed  under 
US  Army  TARDEC  sponsorship.  For  large-scale  simulation  models,  the  total  number  of  simulations  carried  out  for  surrogate  modeling 
could  be  limited  due  to  computation  resource.  In  such  cases  the  inaccuracy  and  uncertainty  of  the  surrogate  model  needs  to  be  quantified. 
For  this,  the  weighted  Kriging  variance  method  is  developed  using  the  corrected  Akaike  information  criterion  (AICc)  to  generate  a 
confidence  interval  of  the  surrogate  model;  and  the  upper  bound  of  the  confidence  interval  is  used  to  obtain  confidence-based  RBDO 
optimum  design  that  satisfies  the  target  reliability.  In  practical  industrial  application,  often  input  data  are  not  sufficient  enough  to  generate 
true  input  distribution  models  for  reliability  analysis  and  RBDO.  When  only  the  limited  input  data  are  provided,  uncertainty  is  induced  on 
the  input  probability  model  and  this  uncertainty  propagates  to  the  reliability  output  which  is  defined  as  the  probability  of  failure.  Thus,  the 
reliability  output  is  considered  to  have  a  probability  distribution  in  this  research,  which  is  obtained  as  a  combination  of  consecutive 
conditional  probabilities  of  input  distribution  type  and  parameters  using  Bayesian  approach.  Using  the  probability  of  the  reliability  output  as 
constraint,  a  confidence-based  RBDO  (C-RBDO)  problem  is  formulated.  For  effective  C-RBDO  process,  the  design  sensitivity  of  the  new 
probabilistic  constraint  is  derived.  As  an  alternative  for  DKG,  a  virtual  support  vector  machine  (VSVM)  is  developed  to  improve  the 
efficiency  of  the  sampling-based  RBDO.  The  VSVM  is  successfully  developed  and  applied  to  the  sampling-based  RBDO  and  demonstrated 
to  be  faster  for  MCS  in  evaluation  of  the  probability  of  failure  and  its  design  sensitivity  compared  to  the  very  accurate  DKG  method  that  the 
research  team  has  developed  in  this  project.  Finally  four  strategies  are  developed  to  improve  the  dynamic  Kriging  (DKG)  method:  (1) 
accurate  correlation  parameter  estimation,  (2)  penalized  maximum  likelihood  estimation  (PMLE),  (3)  correlation  model  selection,  and  (4) 
mean  structure  selection. 
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Scientific  Progress 


SCIENTIFIC  ACCOMPLISHMENTS 


Objective:  Develop  new  technology  and  numerical  methods  for  system-level  reliability-based  design  optimization  (RBDO)  in 
order  to  increase  the  associated  target  confidence  level  under  input  and  simulation  model  uncertainties. 

Approach:  Four  technical  developments  are  pursued: 

1.  For  improved  product  reliability,  design  guidelines  and/or  standards  have  been  recently  modified  to  incorporate  input 
uncertainty  significantly  in  the  modeling  and  simulation  (M&S)  based  design  process.  However,  since  the  reliability-based 
design  optimization  (RBDO)  method  requires  sensitivities  of  performances  with  respect  to  design  and  other  random  input 
variables,  sensitivity  analysis  capability  becomes  a  major  bottleneck  for  broader  use  of  RBDO  methods  in  various  M&S 
applications  of  Army  complex  physical  systems.  To  overcome  this  bottleneck,  during  the  1st  year  of  the  project,  the  sequential 
sampling-based  dynamic  Kriging  (DKG)  method  is  developed  to  generate  the  most  accurate  surrogate  model  in  an  efficient 
manner  compared  with  five  other  popular  surrogate  modeling  methods.  To  make  the  DKG  method  effective  and  efficient  for 
large  scale  complex  applications,  four  parallelization  strategies  have  been  implemented  using  the  parallel  computing  tool  box  in 
Matlab.  The  DKG  method  has  been  integrated  with  the  Iowa  RBDO  (l-RBDO)  software  that  is  developed  under  TARDEC 
sponsorship.  For  successful  application  of  the  sampling-based  RBDO  method  to  the  development  of  reliable  products  with 
desirable  performance,  we  will  develop  a  prediction  method  for  confidence  levels  of  the  simulation  models  and  thus  be  able  to 
certify  the  simulation-based  RBDO  result  with  the  target  confidence  level. 

2.  During  2nd  year  of  the  project,  three  developments  are  made.  First,  to  verify  the  accuracy  of  the  Dynamic  Kriging  (DKG) 
method,  a  comprehensive  comparison  study  has  been  carried  out  with  four  other  popular  surrogate  modeling  methods.  In  this 
comparison  study,  the  DKG  method  is  demonstrated  to  be  the  accurate  method.  Moreover,  the  sequential  sampling  in  SS-DKG 
improved  the  accuracy  of  the  obtained  surrogate  models  further.  Second,  for  practical  engineering  applications  with  large-scale 
simulations,  the  total  number  of  simulations  carried  out  for  surrogate  modeling  is  limited  due  to  computation  resource.  In  such 
cases  the  inaccuracy  and  uncertainty  of  the  surrogate  model  needs  to  be  quantified.  For  this,  the  weighted  Kriging  variance 
method  is  developed  using  the  corrected  Akaike  information  criterion  (AlCc)  to  generate  a  confidence  interval  of  the  surrogate 
model  and  use  the  upper  bound  of  the  confidence  interval  to  have  certain  confidence  that  the  RBDO  optimum  design  satisfies 
the  target  reliability.  Third,  for  such  large-scale  complex  applications,  the  total  computational  time  for  SS-DKG  and  sampling- 
based  RBDO  could  be  large  to  be  unaffordable  if  the  number  of  constraints  is  large.  In  the  platform  developed  in  the  TARDEC 
project,  the  parallel  computing  toolbox  in  Matlab  is  used  to  deploy  the  parallel  computing  strategy. 

3.  In  the  3rd  year,  the  project  was  focused  on  (1)  the  input  model  uncertainty  and  its  propagation  to  reliability  analysis  output 
(probability  of  failure);  and  (2)  virtual  support  vector  machine  (VSVM)  and  efficiency  strategies  for  sampling-based  reliability- 
based  design  optimization  (RBDO).  For  the  1st  research  topic,  often  input  data  are  not  sufficient  enough  for  practical  industrial 
problems  to  generate  true  input  distribution  models  for  reliability  analysis  and  RBDO.  Consequently,  inaccurate  input 
distribution  models  will  lead  to  deficiency  in  design  engineer’s  confidence  in  the  obtained  RBDO  designs.  Therefore,  a 
confidence  concept  needs  to  be  developed  for  the  output.  For  that  purpose,  characterization  of  distribution  of  the  output,  not  an 
output  value,  is  assessed  in  our  research  during  the  third  year.  To  do  this,  dependency  of  the  probability  of  output  is 
decomposed  into  the  probability  of  input  distribution  parameters  and  input  distribution  types.  Bayesian  approach  is  used  for  the 
decomposition,  and  the  correlation  of  input  variables  are  estimated  using  copulas.  For  practical  applications,  Monte  Carlo 
simulation  (MCS)  is  used.  Once  the  confidence  measure  is  established,  a  new  confidence-based  RBDO  problem  is  formulated. 
In  the  confidence-based  RBDO  problem,  not  only  the  target  output  value,  but  also  target  confidence  level  can  be  set  by  user. 
Thus,  the  optimum  design  of  the  confidence-based  RBDO  will  satisfy  the  target  output  with  target  confidence  level;  and  the  user 
can  obtain  a  reliable  optimum  design  even  when  input  data  are  not  sufficient  enough  to  generate  true  input  distribution  models. 
For  the  confidence-based  RBDO  problem,  the  design  sensitivity  analysis  method  for  the  confidence  measure  needs  to  be 
developed.  For  the  2nd  research  topic,  VSVM  is  developed  to  improve  the  efficiency  of  the  sampling-based  RBDO.  The  VSVM 
is  successfully  developed  and  applied  to  the  sampling-based  RBDO  and  demonstrated  to  be  faster  for  MCS  in  evaluation  of  the 
probability  of  failure  and  its  design  sensitivity  compared  to  the  very  accurate  dynamic  Kriging  (DKG)  method  that  the  research 
team  has  developed  in  this  project.  Transformations/Gibbs  sampling  method  is  also  introduced  for  uniform  samples  in  the 
hyper-spherical  local  window.  A  group  sampling  method  has  been  developed,  which  can  replace  the  previous  sequential 
sampling  method  developed  by  the  research  team  for  parallel  computing... 

4.  In  the  4th  year,  the  project  was  focused  on  (1 )  reliability-based  design  optimization  (RBDO)  with  confidence  level  of  output 
(probability  of  failure);  and  (2)  accuracy  improvement  strategies  for  the  Kriging  method  by  using  correlation  model  and  mean 
structure  selections.  For  the  1st  research  topic,  it  is  assumed  that  given  input  data  is  not  sufficient  enough  to  generate  a  true 
input  probabilistic  distribution  for  reliability  analysis  and  RBDO.  Under  the  assumption,  uncertainty  on  the  input  model  will  lead 
a  designer  to  consider  confidence  level  of  the  RBDO  optimum.  To  secure  a  certain  level  of  confidence  level  of  output  at  the 
optimum,  confidence-based  RBDO  (C-RBDO),  which  utilizes  a  distribution  of  output,  not  an  output  value,  has  been  developed. 
C-RBDO  does  not  require  an  input  distribution,  whereas  it  requires  only  input  data.  When  the  input  data  is  given,  C-RBDO 
calculates  probabilities  of  input  parameters  and  candidate  distribution  types  using  Bayesian  approach.  According  to  calculated 
probabilities,  Montel  Carlo  samples  are  generated  and  the  output  is  calculated  at  the  each  sample.  By  doing  this,  the 
distribution  of  output  is  obtained  and  confidence  level  of  output  can  be  acquired  from  the  distribution.  Hence,  a  user  can  obtain 


an  optimum  which  satisfies  target  output  with  target  confidence  ievei  using  C-RBDO.  Furthermore,  design  sensitivity  of  C- 
RBDO  is  deveioped  as  weii  to  achieve  the  optimum  accurateiy  and  efficientiy.  Therefore  a  user  is  abie  to  obtain  a  reiiabie 
optimum  design  effectiveiy  even  when  insufficient  amount  of  data  is  provided.  In  this  year,  the  accuracy  and  efficiency  of  C- 
RBDO  has  been  significantiy  improved  and  C-RBDO  is  tested  with  a  two-dimensionai  mathematicai  exampie.  For  the  2nd 
research  topic,  four  strategies  are  introduced  to  improve  the  dynamic  Kriging  (DKG)  method.  They  are  (1)  accurate  correiation 
parameter  estimation,  (2)  penaiized  maximum  iikeiihood  estimation  (PMLE),  (3)  correiation  modei  seiection,  and  (4)  mean 
structure  seiection.  These  four  accuracy  improvement  strategies  are  combined  into  one  scheme  to  improve  the  accuracy  of  the 
Kriging  method  and  finaiiy  to  reduce  the  number  of  sampies  in  RBDO. 

Scientific  barriers:  Input  variables  which  are  correlated  with  joint  non-Gaussian  CDFs.  Correlations  between  input  variables,  or 
performances  which  are  nonlinear  functions  of  multi-variables,  which  makes  first-order  reliability  method  performance 
unacceptable. 

Significance:  Extend  optimum  design  to  a  variety  of  input  uncertainties  in  material  properties,  geometrical  dimensions,  and 
operational  conditions,  as  well  as  statistical  uncertainties  due  to  limited  test  data  and  lack  of  information. 

Accomplishments: 

1.  Developed  a  new  dynamic  Kriging  (DKG)  method  (probabilities  of  input  parameters  and  candidate  distribution  types 
calculated  by  Bayesian  methods)  to  produce  an  accurate  surrogate  model  (year  1),  shown  to  be  more  accurate  than  existing 
surrogate  methods  (year  2),  and  implemented  on  parallel  HPC. 

2.  Constructed  a  conservative  surrogate  model  using  the  weighted  Kriging  variance  where  the  weight  is  determined  by  the 
relative  change  in  the  corrected  Akaike  Information  Criterion  (AlCc)  of  DKG. 

3.  Established  the  propagation  of  input  model  uncertainty  to  the  reliability  analysis  output  (probability  of  failure).  Identified  the 
probabilities  of  the  distribution  parameters  and  types  using  Bayesian  approach  and  acquired  the  distribution  of  output  using 
MCS.  Analytically  derived  the  design  sensitivity  of  confidence-based  RBDO  (year  3). 

4.  Further  improved  the  accuracy  and  efficiency  of  confidence-based  RBDO  (C-RBDO)  by  replacing  a  finite  difference 
approximation  within  the  analytic  model  by  an  analytic  expression,  making  the  design  sensitivity  of  confidence  level  of  output 
now  fully  analytic.  (4th  year) 

5.  The  correlation  parameter  between  the  input  variables  (as  random  variables)  was  previously  calculated  from  input  data;  it  is 
now  represented  as  a  probability  distribution. 

6.  Computational  complexity  of  C-RBDO  is  greater  than  RBDO  due  to  use  of  confidence  intervals  rather  than  point  estimates 
for  constraints;  this  is  made  less  inefficient  through  several  computational  strategies  (such  as  replacing  some  C-RBDO 
iterations  by  DDO  and  conventional  RBDO  iterations,  using  the  sensitivity  of  these  conventional  iterations  as  a  more  useful 
initial  gradient  for  descent,  using  reusable  MC  simulation  to  replace  NxM  samples  by  N  samples  that  are  mapped  to  M  outputs). 

7.  Developed  VSVM  (virtual  support  vector  machine)  to  improve  efficiency  by  14-30  times  over  DKG  in  MCS  for  reliability 
analysis  at  similar  accuracy. 

8.  Improved  the  accuracy  of  DKG  by  applying  a  genetic  algorithm  to  generalized  pattern  search  for  MLE  in  order  to  decrease 
dependence  on  initial  design,  by  introducing  penalized  MLE  to  improve  performance  in  small  DoE  sample  sizes,  and  by 
selecting  mean  structure  based  on  cross-validation  error  rather  than  by  process  variance. 

Collaborations  and  leveraged  funding:  Scientific  liaison  and  co-publications  (8  archived  journal  papers  and  10  conference 
papers)  with  TARDEC. 

Conclusions:  Accomplishments  of  the  ARO  research  provide  new  computational  method  for  accurately  modeling  distributions 
of  the  correlated  input  variables  using  copula;  confidence  of  the  reliability  of  RBDO  results  when  only  limited  data  is  available  for 
the  input  modeling;  confidence  of  the  reliability  of  RBDO  results  by  constructing  conservative  Kriging-based  surrogate  model; 
providing  an  alternative  sampling-based  RBDO  capability  by  using  a  classification  method,  VSVM.  These  are  new  capabilities 
that  have  been  successfully  published  in  archived  journals. 

Technology  transfer:  Updated  SS-DKG  (ARO  funding).  Copula  (TARDEC  funding),  and  sampling-based  RBDO  (TARDEC 
funding)  with  easy-to-use  user  interface  incorporated  into  the  Pi's  lowa-RBDO  (l-RBDO)  code.  These  are  integrated  in  Iowa 
RBDO  (l-RBDO)  software  system  that  has  been  delivered  to  TARDEC  (Dr.  David  Lamb).  During  the  summer  of  2012,  l-RBDO 
was  installed  on  TARDEC’s  High  Performance  Computing  (HPC)  system  and  Mr.  Nick  Gaul,  who  is  one  of  the  Professor  K.K. 
Choi’s  Ph.D.  students,  worked  as  an  intern  with  TARDEC  scientists  to  successfully  apply  l-RBDO  to  several  Army  application 
problems. 

Pi’s  research  team  assisted  Dr.  David  Lamb  in  TARDEC’s  proposal  for  a  Dedicated  Support  Partition  (DSP)  on  Army  Research 
Laboratory’s  (ARL)  HPC  system,  which  provides  2304  cores  with  4GB/core  and  20.2  Million  CPU  hours  per  year  for  two  years 
to  install  l-RBDO  exclusively  and  demonstrate  applications  for  Army  ground  vehicle  design  problems.  In  June  2013,  the  new 
DSP  was  awarded  to  TARDEC.  In  August  2013,  a  small  company,  RAMDO  Solutions,  LLC,  was  established  and  the  research 
software  l-RBDO  was  delivered  so  that  the  company  can  develop  a  commercial  software  Reliability  Analysis  &  Multidisciplinary 
Design  Optimization  (RAMDO).  RAMDO  will  be  eventually  installed  to  DSP  for  TARDEC’s  use.  For  commercialization,  two 
leading  PIDO  developers,  Altair  and  Dassault  Simulia,  agreed  with  RAMDO  Solutions,  LLC  to  integrate  our  RAMDO  with  their 


FIDO  softwares  -  HyperStudy  and  Isight  -  respectively. 

Technology  Transfer 


